Skip to content

Conversation

@ericharmeling
Copy link
Contributor

@ericharmeling ericharmeling commented May 28, 2020

Related to #6046.
Fixes #3238.

This PR includes:

  • An end-to-end tutorial on developing a basic Spring Boot application on CRDB with Spring Data JDBC for the data access layer.
  • An additional entry in the "Sample Apps" ToC section for "roach-data" apps.
  • Some of the .java files that make up the application project. Those files are included in the tutorial.

The code referenced by this tutorial is located here: https://github.com/cockroachlabs/roach-data/tree/master/roach-data-jdbc. I made a couple minor changes to the application that are reflected in the .java files in this PR and in the tutorial:

  • Removed follower reads from the application. This feature is a little too advanced for a simple tutorial, and is really meant to be used in the context of a multi-region application.
  • Simplified some annotation/imports.

Those changes are reflected in a fork of the repo: https://github.com/ericharmeling/roach-data/tree/no-follower-reads.

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@cockroach-teamcity
Copy link
Member

@cockroach-teamcity
Copy link
Member

@ericharmeling ericharmeling requested a review from kai-niemi May 28, 2020 22:01
@ericharmeling
Copy link
Contributor Author

@kai-niemi
Friendly ping on review :)

@cockroach-teamcity
Copy link
Member

* The main account repository, notice there's no implementation needed since its auto-proxied by
* spring-data.
* <p>
* Should have extended PagingAndSortingRepository in normal cases.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this line of the comment.

import static org.springframework.transaction.annotation.Propagation.MANDATORY;

@Repository
// @Transactional is not needed but here for clarity since we want repos to always be called from a tx context

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// @transactional annotation here to emphasise that repositories should always be called within an existing transaction context

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.


#### Transaction retries

To [avoid transaction contention](performance-best-practices-overview.html#understanding-and-avoiding-transaction-contention), we highly recommend writing [client-side transaction retry logic](transactions.html#client-side-intervention) into applications written on CockroachDB. In this application, transaction retry logic is written into the methods of the `RetryableTransactionAspect` class, which is declared as an aspect with the `@Aspect` annotation. The `@Order` annotation is passed `Ordered.LOWEST_PRECEDENCE-2`, a level of precedence above the primary transaction advisor. This indicates that this advisor must run outside the context of a transaction. Here are the contents of [`RetryableTransactionAspect.java`](https://github.com/cockroachlabs/roach-data/blob/master/roach-data-jdbc/src/main/java/io/roach/data/RetryableTransactionAspect.java):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think retries are a way to handle aborted transactions due to serialization (transient) errors rather than a method for avoiding them. Either through savepoints or retrying local transactions with exponential backoff. At least that's better than propagating transient errors all the way up to the stack to clients/user agents. Ultimately, contention avoidance by schema design and/or reviewing the interleaving of reads/writes is the best option, but not always feasible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

@cockroach-teamcity
Copy link
Member

Copy link

@kai-niemi kai-niemi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really good and detailed explanations of the mechanisms used.


{% include copy-clipboard.html %}
~~~ shell
$ java -jar target/roach-data-jdbc.jar

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another way to use custom connection settings without modifying config files is to just override them via command line:
java -jar target/roach-data-jdbc.jar --spring.datasource.url=jdbc:postgresql://192.168.1.2:26257/roach_data

@cockroach-teamcity
Copy link
Member

@ericharmeling
Copy link
Contributor Author

@kai-niemi
I just added a section on Liquibase. Please review when you get the chance. Thanks! :)

@vy-ton
Copy link
Contributor

vy-ton commented Jun 22, 2020

Is it worth renaming the roach-data repo to something like java-samples? Eric, we should merge your roach-data changes?

If the code sample uses Liquibase, can we add beta support here?

@ericharmeling
Copy link
Contributor Author

Is it worth renaming the roach-data repo to something like java-samples?

I don't think we should rename the app to something as generic, for a few reasons:

  • These apps are Spring apps, not just Java apps, which is an important distinction a little muddled by having "java" in the project name.
  • There are other apps that we use for docs and testing that are written in Java (e.g. https://github.com/cockroachlabs/hello-world-java-jdbc and https://github.com/cockroachdb/examples-orms/tree/master/java, to name two). Naming this project "java-samples" could cause some confusion with those samples.
  • This app is slightly more complex than a Hello World tutorial (like the Spring MyBatis tutorial), but not as complex as our multi-region app, or Kai's roachbank app. Even if we were to name the app "spring_samples", I think that distinction of complexity would also be muddled.
  • There is a possibility that we mimic this application's design in other languages. A project name like "Roach Data" would be easier to port to other languages.

Eric, we should merge your roach-data changes?

Kai already reviewed the changes I list above, and I merged them. I do have another PR open (cockroachlabs/roach-data#2), but it's not critical.

If the code sample uses Liquibase, can we add beta support here?

Does Liquibase fit our working definition of "beta" support? If not, we might be able to add another support level (e.g., "Experimental) until it does. Customers have been asking about Liquibase for a while now, and I know that many people actually use Liquibase with CRDB without issue.

@vy-ton
Copy link
Contributor

vy-ton commented Jun 22, 2020

I don't think we should rename the app to something as generic, for a few reasons:

Reasons make sense to me. As we increase the number of code samples in a specific language, let's think about how we can connect the dots for an AppDev

Does Liquibase fit our working definition of "beta" support? If not, we might be able to add another support level (e.g., "Experimental) until it does.

I think current Liquibase support matches the beta defintion here. It'll be a good exercise to create a checklist like this for schema migration tools. Personally, I'm not a fan of Experimental because I'm not sure what the distinction would be with Beta. To me, one reason to add a 3rd support level would be to indicate tools we're aware of but have no intention of supporting. Liquibase, however, doesn't fall into that cateogory

@ericharmeling
Copy link
Contributor Author

I don't think we should rename the app to something as generic, for a few reasons:

Reasons make sense to me. As we increase the number of code samples in a specific language, let's think about how we can connect the dots for an AppDev

Sounds good!

Does Liquibase fit our working definition of "beta" support? If not, we might be able to add another support level (e.g., "Experimental) until it does.

I think current Liquibase support matches the beta defintion here. It'll be a good exercise to create a checklist like this for schema migration tools. Personally, I'm not a fan of Experimental because I'm not sure what the distinction would be with Beta. To me, one reason to add a 3rd support level would be to indicate tools we're aware of but have no intention of supporting. Liquibase, however, doesn't fall into that cateogory

Awesome. I'll open a PR to update the tooling page, and then I'll update this PR to link to the Schema Migration Tools section on that page.

Copy link

@kai-niemi kai-niemi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@kai-niemi
Copy link

I think Liquibase qualifies for beta level support. It doesn't implement retry logic, so in some cases having to re-run the changesets are needed (app launch fails in that case). This app also use a very limited part of the feature set.

@cockroach-teamcity
Copy link
Member

@cockroach-teamcity
Copy link
Member

@ericharmeling
Copy link
Contributor Author

@kai-niemi

Worth mentioning perhpas that spring profiles can be used to run different Liquibase changesets for testing or different environments. In that case its only a matter of linking the same name of the spring profiles with the Liquibase 'contexts'. There's only one profile+context in this application for simplicity.

I just added a commit that includes a couple paragraphs about Spring profiles and Liquibase contexts.

I think Liquibase qualifies for beta level support. It doesn't implement retry logic, so in some cases having to re-run the changesets are needed (app launch fails in that case). This app also use a very limited part of the feature set.

That commit also adds Spring to the list of third-party tools. We point to the drivers/ORMs for support level.

@ericharmeling ericharmeling requested a review from rmloveland June 25, 2020 16:04
@ericharmeling
Copy link
Contributor Author

@kai-niemi
Thank you so much for your reviews, Kai. I'm moving this to writer review.

Copy link
Contributor

@rmloveland rmloveland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Left a few comments about small issues.


## Step 6. Run the application

Compiling and running the application code will start a web application, initialize the `accounts` table in the `roach_data` database, and submit some simple requests to the app's REST API that result in [atomic database transactions](transactions.html) on the running CockroachDB cluster. For details about the application code, see [Implementation details](#implementation-details).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest "simple requests" -> "requests"

(per ye olde Style Guide )

twitter: false
---

This tutorial shows you how to build a simple [Spring Boot](https://spring.io/projects/spring-boot) web application with CockroachDB, using the [Spring Data JDBC](https://spring.io/projects/spring-data-jdbc) module for data access. The code for the example application is available for download from [GitHub](https://github.com/cockroachlabs/roach-data/tree/master), along with identical examples that use JPA, jOOQ, and MyBatis for data access.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest removing "simple" per style guide (link to reference is in another comment).

@@ -0,0 +1,834 @@
---
title: Build a Spring App with CockroachDB and JDBC
summary: Learn how to use CockroachDB from a simple Spring application with the JDBC driver.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/simple//g

twitter: false
---

This tutorial shows you how to build a simple [Spring Boot](https://spring.io/projects/spring-boot) web application with CockroachDB, using the [Spring Data JDBC](https://spring.io/projects/spring-data-jdbc) module for data access. The code for the example application is available for download from [GitHub](https://github.com/cockroachlabs/roach-data/tree/master), along with identical examples that use JPA, jOOQ, and MyBatis for data access.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest linking "JPA", "MyBatis" etc., so if users get here from search but decide they actually want those other things, it's a click away.


## Step 6. Run the application

Compiling and running the application code will start a web application, initialize the `accounts` table in the `roach_data` database, and submit some simple requests to the app's REST API that result in database transactions on the running CockroachDB cluster. For details about the application code, [see below](#implementation-details).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/simple//g


{% include copy-clipboard.html %}
~~~ shell
$ curl -X GET http://localhost:8080/account | json_pp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very "nit", but perhaps add a note to the UNIX-naive user mentioning json_pp (or alternatives like jq or so?)

}
~~~

The `http://localhost:8080/transfer` endpoint performs transfers between accounts. `POST` requests to this endpoint are executed as writes (i.e., `INSERTS` and `UPDATES`) to the database.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could link to INSERT and UPDATE. Plurals should be formatted as INSERTs, I think.


When the application is started, all of the queries specified by the changesets are executed in the order specified by their `changeset` tag's `id` value. At application startup, Liquibase also creates a table called [`databasechangelog`](https://docs.liquibase.com/concepts/databasechangelog-table.html) in the database where it performs changes. This table's rows log all completed changesets.

To see the completed changsets after starting the application, open a new terminal, start the [built-in SQL shell](cockroach-sql.html), and query the `databasechangelog` table:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo in "changsets"

When [transaction management](https://docs.spring.io/spring/docs/current/spring-framework-reference/data-access.html#transaction-declarative) is enabled in an application, Spring automatically wraps all objects annotated with `@Transactional` in [a proxy](https://docs.spring.io/spring/docs/current/spring-framework-reference/core.html#aop-understanding-aop-proxies) that handles calls to the object. By default, this proxy starts and closes transactions according to the configured transaction management behavior (e.g., the `propagation` level).

Using [@AspectJ annotations](https://docs.spring.io/spring/docs/current/spring-framework-reference/data-access.html#transaction-declarative-aspectj), this sample application extends the default transaction proxy behavior with two other explicitly-defined [aspects](https://en.wikipedia.org/wiki/Aspect_(computer_programming)): `TransactionHintsAspect` and `RetryableTransactionAspect`. Methods of these aspects are declared as [advice](https://en.wikipedia.org/wiki/Advice_(programming)) to be executed around method calls annotated with `@Transactional`.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might (?) be worth being a bit more explicit here about listing and linking the subsections since this doc is pretty long, e.g.,

For more information, see the following sections:


Transactions may require retries if they experience deadlock or [transaction contention](performance-best-practices-overview.html#understanding-and-avoiding-transaction-contention) that cannot be resolved without allowing [serialization](demo-serializable.html) anomalies. To handle transactions that are aborted due to transient serialization errors, we highly recommend writing [client-side transaction retry logic](transactions.html#client-side-intervention) into applications written on CockroachDB.

In this application, transaction retry logic is written into the methods of the `RetryableTransactionAspect` class, declared an aspect with the `@Aspect` annotation. The `@Order` annotation on this aspect class is passed `Ordered.LOWEST_PRECEDENCE-2`, a level of precedence above the primary transaction advisor. This indicates that the transaction retry advisor must run outside the context of a transaction. Here are the contents of [`RetryableTransactionAspect.java`](https://github.com/cockroachlabs/roach-data/blob/master/roach-data-jdbc/src/main/java/io/roach/data/RetryableTransactionAspect.java):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe "..., and declared an aspect" ?

Copy link
Contributor Author

@ericharmeling ericharmeling left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR @rmloveland !

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ericharmeling, @kai-niemi, @rmloveland, and @transactional)


v20.1/build-a-spring-app-with-cockroachdb-jdbc.md, line 696 at r4 (raw file):

Previously, kai-niemi (Kai Niemi) wrote…

Worth mentioning perhpas that spring profiles can be used to run different Liquibase changesets for testing or different environments. In that case its only a matter of linking the same name of the spring profiles with the Liquibase 'contexts'. There's only one profile+context in this application for simplicity.

Done.


v20.1/build-a-spring-app-with-cockroachdb-jdbc.md, line 3 at r6 (raw file):

Previously, rmloveland (Rich Loveland) wrote…

s/simple//g

Done.


v20.1/build-a-spring-app-with-cockroachdb-jdbc.md, line 8 at r6 (raw file):

Previously, rmloveland (Rich Loveland) wrote…

Suggest linking "JPA", "MyBatis" etc., so if users get here from search but decide they actually want those other things, it's a click away.

Done.


v20.1/build-a-spring-app-with-cockroachdb-jdbc.md, line 132 at r6 (raw file):

Previously, rmloveland (Rich Loveland) wrote…

Suggest "simple requests" -> "requests"

(per ye olde Style Guide )

Done.


v20.1/build-a-spring-app-with-cockroachdb-jdbc.md, line 212 at r6 (raw file):

Previously, rmloveland (Rich Loveland) wrote…

s/simple//g

Done.


v20.1/build-a-spring-app-with-cockroachdb-jdbc.md, line 258 at r6 (raw file):

Previously, rmloveland (Rich Loveland) wrote…

Love the cowsay!

:)


v20.1/build-a-spring-app-with-cockroachdb-jdbc.md, line 352 at r6 (raw file):

Previously, rmloveland (Rich Loveland) wrote…

s/simple//g

Done.


v20.1/build-a-spring-app-with-cockroachdb-jdbc.md, line 356 at r6 (raw file):

Previously, rmloveland (Rich Loveland) wrote…

This is very "nit", but perhaps add a note to the UNIX-naive user mentioning json_pp (or alternatives like jq or so?)

Added a little sentence about the commands.


v20.1/build-a-spring-app-with-cockroachdb-jdbc.md, line 457 at r6 (raw file):

Previously, rmloveland (Rich Loveland) wrote…

Could link to INSERT and UPDATE. Plurals should be formatted as INSERTs, I think.

Done.


v20.1/build-a-spring-app-with-cockroachdb-jdbc.md, line 657 at r6 (raw file):

Previously, rmloveland (Rich Loveland) wrote…

typo in "changsets"

Done.


v20.1/build-a-spring-app-with-cockroachdb-jdbc.md, line 774 at r6 (raw file):

Previously, rmloveland (Rich Loveland) wrote…

It might (?) be worth being a bit more explicit here about listing and linking the subsections since this doc is pretty long, e.g.,

For more information, see the following sections:

Done.


v20.1/build-a-spring-app-with-cockroachdb-jdbc.md, line 805 at r6 (raw file):

Previously, rmloveland (Rich Loveland) wrote…

maybe "..., and declared an aspect" ?

Updated the wording.

@cockroach-teamcity
Copy link
Member

@ericharmeling ericharmeling merged commit c940a43 into master Jun 29, 2020
@ericharmeling ericharmeling deleted the spring-roach-data branch August 24, 2020 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Retry example for Spring

5 participants